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(57) A video signal compressor, includes ap- 
paratus (11-17) for generating motion vectors 
for respective blocks of video data. Ones of the 
motion vectors (other than those which provide 
(18) direction to co-located blocks in other 
frames) are determined to be candidates to 
conversion to vectors which do provide direc- 
tion to co-located blocks. However, conversion 
is precluded (100,102) if the vector is deter- 
mined to be correlated with one of the 
neighboring motion vectors. 
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This invention relates to motion compensated 
compression apparatus for processing video data on 
a block by block basis and more particularly for proc- 
essing motion vectors associated with respective 
blocks. 

Current trends in video signal recording and 
transmission are leaning toward the use of digital 
compression techniques. Toward this end the Interna- 
tional Organisation for Standardisation has been de- 
veloping a series of compression signal protocols 
designated MPEG1, MPEG2, etc. These protocols re- 
quire that the video signal be compressed according 
to motion compensated prediction techniques in 
which the video signal is divided into matrices or 
blocks of contiguous pixels and thereafter the signal 
is processed on a block basis. 

In typical motion compensated predictive com- 
pression successive images are predicted from prior 
compressed images. The current image is subtract- 
ed, on a pixel by pixel basis, from its associated pre- 
dicted image, and the differences or residues are en- 
coded for storage or transmission. In generating the 
predicted images, blocks of the current image are 
compared with like sized blocks of the predicted im- 
age until the closest block match is found. A set of mo- 
tion vectors is generated which indicate the block of 
the predicted image which most nearly matches the 
real image. The differences between the two blocks 
and the motion vectors form, in major part, the com- 
pressed information. The residues may however un- 
dergo transform processing and variable length and 
statistical encoding. For detailed information on mo- 
tion compensated predictive compression see Peter 
A. Ruetzetal. "A High-Performance Full-Motion Vid- 
eo Compression Set", or Hiroshi Fujiwara, etal "An 
All-ASIC Implementation of a Low Bit-Rate Video Co- 
dec" and Chaur-Heh Hsieh, et.al. "VLSI Architecture 
for Block-Matching Motion Estimation Algorithm" all 
of which are to be found in IEEE TRANS. ON CIR- 
CUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, 
VOL 2, No. 2, JUNE 1992. 
In certain systems such as the MPEG 1 system, a de- 
cision is made whether to code a set of motion vectors 
or not. This is called a MC/no-MC decision. 

The purpose of the MC/no-MC decision is to de- 
termine for each block of compressed data whether to 
select the motion vector(s) resulting from the motion 
estimation search (MC), or to select (0,0) motion vec- 
tors (no-MC) instead. The MC/no-MC decision is ap- 
plied to each block of the Picture. If the particular sys- 
tem performs both forward and backward prediction 
as in the MPEG systems a MC/no-MC decision is 
made independently for both forward and backward 
motion vectors. 

The no-MC motion vector should be used in those 
cases where the MC motion vector is useless (it car- 
ries no information) and is costly to encode, that is, it 
tends to create an effect of swarming noise, and 
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makes error concealment techniques less efficient. 
This situation exists particularly in flat image areas, 
like a sky, or a wall, etc, where the MC motion vectors 
tend to be totally random and are the result of random 

5 noise present in the source pictures during the motion 
estimation search, and not due to scene changes. 

Typical MC/no-MC decision algorithms are based 
on prediction errors. Prediction errors can be the 
MAE (Mean Absolute Error per pixel) or the MSE 

10 (Mean Square Error per pixel) between the predicted 
block and the target block. Nominally a no-MC deci- 
sion is made if the spatially collocated predicted and 
target blocks yield the smallest prediction error. In ad- 
dition, if the smallest prediction error generated in a 

15 block matching search, is produced by the target 
block and a non collocated block, and this prediction 
error is less than the prediction error of the collocated 
target and predicted blocks by only a predetermined 
threshold, a no-MC decision is made. The optimum 

20 threshold needs to be determined by experiment for 
respective systems. It has been found however that 
if the MSE is used as a prediction error a threshold of 
1 provides good results. 

If a no-MC decision is made for a particular block 

25 this implies a motion vector set of (0, 0). This vector 
set can be encoded as a vector or the block of residue 
data can be flagged as a no-MC block without the 
necessity of including the vector, thereby enhancing 
compression efficiency. 

30 The foregoing MC/no-MC decision algorithm pro- 
duces a very annoying side effect, called herein the 
"dirty window effect". It appears like dirt sticking to 
the screen in flat areas (where no-MC motion vectors 
are used), and can be very obvious and disturbing. 

35 The cause of the problem is that the no-MC motion 
vectors are used in areas almost flat, but containing 
in fact a very low intensity pattern such as a blurred 
background, or a cloud for example. This problem can 
be solved by lowering the value of the threshold used 

40 in the decision, but then another problem appears. 
The prediction error caused by noise becomes larger 
than the threshold and no-MC motion vectors will sel- 
dom be used, causing what can be described as a 
swarming noise effect. 

45 The present invention is directed to apparatus 
which takes advantage of MC/no-MC decisions with- 
out incurring the undesirable "dirty window" or 
"swarming noise effects." 

The present invention comprises a motion vector 

50 producing apparatus for generating motion vectors 
indicative of the relative motion of respective blocks 
of video data between image intervals. This appara- 
tus includes circuitry for performing block matching to 
produce the respective motion vectors, and appara- 

55 tus for performing a MC/no-MC decision based upon, 
for example, prediction errors. A plurality of motion 
vectors for a corresponding plurality of neighboring 
blocks are concurrently made available. Motion vec- 
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tors associated with respective blocks are tested for 
correlation with motion vectors associated with neigh- 
boring blocks. Motion vectors for which no correlation 
exists and for which a MC to no-MC decision has been 
made are then converted to zero-valued motion vec- 
tors. 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIGURE 1 is a block diagram of a video signal 
compression apparatus including motion vector gen- 
erating apparatus embodying the present invention. 

FIGURE 2 is a flow chart depicting in part the op- 
eration of the motion vector generating apparatus of 
FIGURE 1. 

FIGURE 3 is a flow chart of the Correlation Deci- 
sion block of FIGURE 2. 

FIGURES 4, 5 and 6 are vector diagrams useful 
in describing the invention 

This present MC/no-MC processor was designed 
in order to avoid selecting no-MC motion vectors in 
situations where it would cause the dirty window ef- 
fect. It has been observed in various scenes where 
the dirty window effect was present, that the motion 
vector field was quite correlated even in the areas 
where no-MC motion vectors were selected. On the 
other hand, in the areas where no-MC motion vectors 
were selected and no dirty window effect was visible, 
the motion vector field appeared very random. Fol- 
lowing this observation, it was deduced that the cor- 
relation of the motion vector field around each block 
of data was a good indication that the MC motion vec- 
tors which were changed to no-MC vectors were in 
fact significant, and that the no-MC motion vectors 
should not be selected in this case. 

An exemplary correlation criterion which has 
been found to provide favorable results is the follow- 
ing. A MC motion vector is considered as correlated 
if it is almost equal to at least one of the motion vec- 
tors of a neighboring block. Neighboring blocks are 
those blocks of video data located immediately 
above, under, on the right and on the left of the con- 
sidered block. There are normally 4 neighboring 
blocks, except for those blocks located on the edge or 
the corner of the picture, where there are only 3 or 2 
neighboring blocks. By almost equal is meant that the 
absolute differences of both the x and y coordinates 
of the motion vectors, expressed in half-pel units, are- 
less or equal to a threshold (a typical threshold is 2). 
If the MC motion vector is correlated, then the MC mo- 
tion vector is always selected. If the MC motion vector 
is not correlated, then a MC/no-MC decision is made 
according to the relative values of the prediction er- 
rors. 

In order to implement this new MC/no-MC deci- 
sion process, it is necessary to memorize or store the 
motion vectors of 3 rows of blocks of video data, and 
the motion vectors must be computed one row of 



blocks in advance of making the MC/no-MC decision 
based on correlation. However, it is possible to sim- 
plify hardware implementation by using a simplified 
version of this MC/no-MC decision process. In the 

5 simplified version, the only neighboring blocks that 
are considered are the ones located above and on the 
left of the current block, and possibly the one to the 
right, since this involves only a very little additional 
delay. Such a simplified version, although not as ef- 

w fective requires significantly less memory and is a 
good compromise in terms of cost-to- benefit. 

FIGURE 4 illustrates the preferred approach 
wherein the respective blocks represent blocks of vid- 
eo data corresponding to respective contiguous im- 

15 age areas. A portion of three rows of blocks is shown, 
with the block designated C being the current block 
under consideration. The arrows represent motion 
vectors for the respective blocks. In the preferred em- 
bodiment, the motion vector of block C is compared 

20 with the motion vectors of blocks T, B, R and L This 
requires the storage of motion vectors for at least two 
rows plus one blocks of data. 

FIGURE 5 represents a simplified approach 
which requires the storage of motion vectors for one 

25 row of blocks. FIGURE 5 suggests several approach- 
es, all of which are viable and produce acceptable re- 
sults. One of these approaches is to determine cor- 
relation between vector V c and vectors V L and V T . An- 
other approach is to check for correlation between 

30 vector V c and vectors V T . , V Tf V T+ , and V L . Another 
approach is to check for correlation between vectors 
V c . and only V L . This latter approach requires sub- 
stantially no data storage and provides an improve- 
ment over just performing the MC/no-MC decision us- 

35 ing the prediction errors. 

FIGURE 6 indicates in part the method of vector 
correlation. Each vector is presumed to be defined in 
relative terms describing the differences in coordin- 
ates between the location of the target and the pre- 

40 dieted blocks in x and y component values. A differ- 
encing apparatus determines the magnitude of the 
differences Ax and Ay between the vector V c and the 
neighboring vectors VY If both of the absolute differ- 
ences I Axi I and I Ay ( , I determined for a neighboring 

45 vector is less than a threshold, the vector V c is 
deemed to be correlated with the vector V|, and a 
change of a MC vector to a no-MC vector is disallowed 
for the block C. 

Note, in the foregoing discussion the video data 

so has been represented as being in blocks, and the mo- 
tion vectors associated with respective blocks. In the 
MPEG signal protocol, a plurality of blocks of data are 
arranged into macroblocks, with a single motion vec- 
tor associated with a macrobock. In terms of the dis- 
ss cussion, with systems such as MPEG, the term mac- 
roblock of data is interchangeable with the term block 
of data. 

FIGURE 1 illustrates exemplary apparatus for 
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practicing the invention. In FIGURE 1 the elements 
10 through 18 typify known motion compensated pre- 
dictive video signal compression systems. Ignoring 
for the moment elements 1 00-102, video signal is ap- 
plied at terminal 10 and is coupled to one input of a s 
subtracter 11. A video signal representing a predicted 
image provided by the predictor 1 7 is applied to a sec- 
ond input terminal of the subtracter 11 , which in turn 
develops image differences or residues. The resi- 
dues are applied to a discrete cosine transform appa- 10 
ratus 12 which transforms the residues to coeffi- 
cients representing spatial frequency. The coeffi- 
cients are quantized by a quantizer 13, and applied 
to run length and statistical encoder 19. The quan- 
tized, run length and statistically encoded data is ap- 15 
plied to a signal formatter 14, which arranges the en- 
coded video data and synchronizing signal for storage 
or transmission. 

The quantized data from element 13 is applied to 
the cascade combination of an inverse quantizer 15 20 
and an inverse discrete cosine transformer 16, which 
perform the inverse functions of elements 13 and 12 
respectively. The output signal provided by the in- 
verse discrete cosine transformer 16 corresponds to 
the residues provided by the subtracter 1 1 . These re- 25 
sidues are coupled to a motion compensated predic- 
tor 17 which nominally contains at least one frame of 
memory. The memory contains the last predicted im- 
age. Corresponding blocks of data from the memory 
in the predictor 17 and the inverse discrete cosine 30 
transformer 16 are summed on a pixel basis to gen- 
erate a new predicted image. 

A motion vector generator 18 generates motion 
vectors according to known block matching techni- 
ques. The motion vectors are coupled to the signal 35 
formatter 14, wherein they are multiplexed with the 
blocks of video data to form a compressed data 
stream. 

In the present invention, the motion vectors are 
caused to undergo the further correlation test before 40 
being applied to the formatter 14. To accomplish the 
correlation test, the motion vectors for respective 
blocks are stored within a memory 100. The vector 
generator 18 performs a MC/no-MC decision based 
upon prediction errors, but does not convert any MC 45 
vectors to no-MC vectors. Rather it generates a flag 
indicating that a particular MC vector is a candidate 
to be converted to a no-MC vector. These flags are 
stored with the associated motion vectors in the mem- 
ory 1 00. If the system is arranged to correlate vectors 50 
over three rows of blocks as indicated in FIGURE 4, 
the memory 1 00 must include enough storage to store 
the vectors and flags for two rows of blocks. Alterna- 
tively if the system is arranged to perform vector cor- 
relation over two rows of blocks, then the memory 55 
100 must include enough storage space to store one 
row of vectors and associated flags. 

Flags for respective blocks are sequentially ac- 



cessed from the memory 100 by the MC/no-MC de- 
cision element 102. If a flag indicates that a block is 
a candidate for MC to no-MC conversion, the decision 
block 102 performs a vector correlation decision. 
Then, if the respective candidate MC vector is deter- 
mined to be non correlated with any vector associated 
with a neighboring block, the MC vector is converted 
or transformed to a no-MC vector. 

FIGURE 2 is a flowchart describing the operation 
of the motion vector generator 18. The generator is re- 
set [200] at the beginning of each video field/frame. 
A block index K is set to zero [201] and the system 
then fetches [202] the first block. A block matching 
process is performed [203] in order to generate vec- 
tors for the block K. The block matching prediction er- 
rors are examined [204] to determine if the vectors for 
the block K are candidates for a MC vector to no-MC 
vector conversion. If the vectors are candidates for 
conversion a flag is set [207] to 1 and if not the flag 
is set [206] to 0. The vectors for block K and the as- 
sociated flag is then stored [208] in a memory. A cor- 
relation decision is made [209] to determine if the sur- 
rounding vectors are correlated and if the flag is 1 or 
0, and to convert or not convert the MC vector to a no- 
MC vector. The index K is then incremented [210] and 
the incremented index checked [211] to determine if 
all the blocks in the current frame have been process- 
ed. If they have the system returns to step [200], and 
if not to step 201. 

FIGURE 3 shows the processes included in the 
correlation decision step [209]. Depending upon the 
location of a block within a frame it may or may not 
be located in a position amenable to vector correla- 
tion determination. For example, if the current block 
under examination is in the topmost row of blocks of 
a frame, there are no blocks above and hence there 
are no vectors with which to be conveniently correlat- 
ed. The first step [300] of the correlation process is 
therefor to determine if the current block is within a 
frame area for which it makes sense to do a vector 
correlation. In the simplest mode this may be to de- 
termine if the block number is large enough to be a 
block of the second row and small enough to be within 
the frame. 

Once this determination is made the flag associ- 
ated with block K is accessed from memory [301] and 
examined [302] to determine whether it is a 1 or 0. If 
it is a zero (not a candidate for MC/no-MC conversion) 
nothing need be done and the vectors determined at 
step [203] are forwarded [311] for utilization in the 
compression process. On the other hand if the flag is 
a one, an index i is set to n [303] where n may be equal 
to the number of blocks in a row of blocks. Vectors for 
blocks K and K + i are accessed [304] from memory. 
A correlation test is performed [305] between the vec- 
tors from blocks K and K + L If there is correlation be- 
tween the vectors, the vectors associated with block 
K are not converted to no-MC vectors, and the original 
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vectors are forwarded [311] for utilization in the com- 
pression process. If the vectors from blocks K and K 
+ i are not correlated, the index i is examined [306] to 
determine if all neighboring vectors have been tested. 
If they have the motion vectors for block K are con- 5 
verted [309] to no-MC vectors, and these are forward- 
ed [310] for use in the compression process. If all of 
the neighboring vectors have not been tested, the in- 
dex i is appropriately changed [307] to identify the 
next neighboring vector to be accessed for correlation 10 
testing at step [304]. It should be appreciated that the 
index i will be changed in a manner to accommodate 
blocks occurring at the edges of respective frames 
and to identify the appropriate blocks for which vec- 
tors are to be tested, as for example blocks T, L, R, is 
and B when considering block C in FIGURE 4. 

In certain compression systems, if the motion 
vectors for a particular block indicate that the target 
block that most nearly matches the predicted block is 
co-located therewith, the coding of the block may be 20 
designated as non motion compensated and no mo- 
tion vectors may be associated therewith. This has 
the same affect as providing a zero-valued motion 
vector. Hence, in the claims that follow, it is intended 
that conversion of motion vectors to zero-valued mo- 25 
tion vectors should be considered to be equivalent to 
changing a motion compensated block of data to a 
non motion compensated coded block with or without 
motion vectors. 



3. The apparatus set forth in claim 1 characterized 
in that said motion vectors include orthogonal 
components, and said means for determining if 
vectors are correlated includes; 

means for determining if the magnitude of 
the differences between the orthogonal compo- 
nents of the motion vector (MC) and the orthogo- 
nal components of a motion vector of a neighbor- 
ing block are both less than a predetermined val- 
ue. 

4. The apparatus set forth in claim 1 characterized 
in that said means for determining if motion vec- 
tors are correlated includes means for performing 
a correlative determination with motion vectors 
associated with a plurality of neighboring blocks, 
and converting said candidate motion vector 
(MC) if correlation is not detected with respect to 
a motion vector associated with any one of said 
plurality of neighboring blocks. 

5. In video signal compression apparatus of the 
type which processes video data on a block basis 
and generates motion vectors associated with re- 
spective blocks via a block matching process, 
characterized by apparatus for determining if mo- 
tion vectors from neighboring blocks are correlat- 
ed, and converting ones of the motion vectors 
which are not correlated to zero-valued motion 
vectors. 



Claims 



1. In video signal compression apparatus of the 
type which processes video data on a block basis 35 
and generates motion vectors associated with re- 
spective blocks via a block matching process, 
motion vector processing apparatus character- 
ized by; 

means (1 8) responsive to parameters gen- 40 
erated during said block matching process for de- 
termining which of respective non-zero motion 
vectors (MC) are candidates for conversion to 
zero valued motion vectors (no-MC); and 

means (100,102) for determining if motion 45 
vectors associated with blocks in the neighbor- 
hood of blocks associated with candidate motion 
vectors (MC) are correlated, and if such vectors 
are not correlated, converting said candidate mo- 
tion vectors (MC) to zero valued motion vectors so 
(no-MC). 



2. The apparatus set forth in claim 1 characterized 
in that said apparatus includes: 

means for storing motion vectors associat- 55 
ed with respective blocks; and 

means for concurrently accessing motion 
vectors from a plurality of blocks. 
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